Skip to content

创建统一的参数配置基类#293

Open
MoringLotus wants to merge 16 commits intoInfiniTensor:mainfrom
MoringLotus:qy_interview
Open

创建统一的参数配置基类#293
MoringLotus wants to merge 16 commits intoInfiniTensor:mainfrom
MoringLotus:qy_interview

Conversation

@MoringLotus
Copy link
Copy Markdown

@MoringLotus MoringLotus commented Apr 8, 2026

  1. 当前工作： 统一所有的parser到base_config中(python/infinilm/base_config.py)， 支持通过infinilm.base_config来使用配置类
  2. 下一步开展对args的处理，对现有的几个代表性的script使用base_config进行简化

@MoringLotus MoringLotus marked this pull request as ready for review April 8, 2026 07:49
@MoringLotus MoringLotus requested a review from a team April 8, 2026 07:49
wooway777
wooway777 requested changes Apr 9, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@wooway777 wooway777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

大致上可以接着写，有一些参数的可选项相对于之前的版本有变化，需要更新/补充。

然后后面应该先接着这个pr继续写，替换掉目前四个入口里的argument parsing，以可以跑通的单位合并。

Comment thread README.md
- 编译并安装 `InfiniLM`

```bash
xmake && xmake install
Copy link
Copy Markdown
Collaborator

@wooway777 wooway777 Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个为什么要删？

Comment thread jiuge.sh Outdated
Copy link
Copy Markdown
Collaborator

@wooway777 wooway777 Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

应该不需要加这个文件，不然不是又多一个入口

Copy link
Copy Markdown
Author

@MoringLotus MoringLotus Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的，会在后面PR中移除该文件和上面README.md的修改

Comment thread python/infinilm/base_config.py Outdated
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "../../scripts"))
from libinfinicore_infer import DeviceType
Copy link
Copy Markdown
Collaborator

@wooway777 wooway777 Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

尽量不要引用这个，infinicore infer是旧版用的，新的尽量不要依赖。

wooway777
wooway777 reviewed Apr 13, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@wooway777 wooway777 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

以及多了个conflict，需要rebase然后resolve一下

Comment thread examples/jiuge.py Outdated
wooway777
wooway777 reviewed Apr 13, 2026
View reviewed changes
Comment thread python/infinilm/base_config.py Outdated
Comment thread python/infinilm/base_config.py Outdated
wooway777
wooway777 reviewed Apr 14, 2026
View reviewed changes
Comment thread python/infinilm/base_config.py Outdated
@MoringLotus
Copy link
Copy Markdown
Author

MoringLotus commented Apr 15, 2026

修改内容：
python/infinilm/server/inference_server.py 解决ignore_eos冲突,
examples/jiuge.py 添加空行清除特殊符号，
python/infinilm/base_config添加了ignore_eos选项，添加了attn选择逻辑，解决可能发生的paged attn无法执行问题
移除了注释和一些无用代码

测试用启动脚本：
python python/infinilm/server/inference_server.py --device nvidia --model="model_weight" --max_tokens=2048 --max_batch_size=8 --ignore_eos --enable-paged-attn

python examples/bench.py --model "model_weight" --enable-paged-attn --batch-size 4 --input-len 1024 --output-len 512 --device nvidia --warm_up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wooway777 wooway777 wooway777 requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MoringLotus @wooway777